`ultralytics-inference 0.0.58` Add scene text OCR with YOLO26-Text and PaddleOCR by onuralpszr · Pull Request #631 · ultralytics/inference

onuralpszr · 2026-11-10T09:58:59Z

Add TextRecognizer and predict --ocr to read text with YOLO26-Text and PaddleOCR, like Python solutions.OCR
Work with box, oriented box and segmentation detectors, or a custom recognizer function
Download the OCR models automatically from the assets release
Draw OCR results like Python, with --line-width, --show-labels and --show-conf
Same texts as Python on CPU, CUDA, TensorRT and OpenVINO (one small difference on the l model)
Bump version to 0.0.58

🛠️ PR Summary

🌟 Summary

Version 0.0.58 adds native scene-text OCR using YOLO26-Text for text detection and PaddleOCR for recognition, exposed through both a Rust API and predict --ocr.

📊 Key Changes

Added TextRecognizer to read detections from boxes, oriented boxes, or segmentation masks; custom recognition functions are also supported. Results include recognized text, confidence, detection confidence, and polygon, and are returned in reading order.
Added --ocr, which defaults to yolo26n-text.onnx when no model is specified; the YOLO26-Text and PaddleOCR model names are included in model auto-download handling.
OCR crops are orientation-checked and recognized in batches. OCR annotations draw text polygons and labels, with --line-width, --show-labels, and --show-conf applied.
Added companion-model execution-provider handling for OCR and made the tensorrt feature include cuda; bumped the Rust crate, web crate, and npm package versions to 0.0.58.

🎯 Purpose & Impact

Users can run scene-text detection and recognition from the CLI or Rust API, including with box, oriented-box, and segmentation results. The diff documents OCR as native-only; it is not included in the WebAssembly package.

📋 Skipped 2 files (lock files, generated, images, etc.)

Cargo.lock
web/package-lock.json

…ch Python pixel for pixel Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

…y default like Ultralytics Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

…ltralytics Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

…tics scale_coords Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

…zer and predict --ocr Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

…d reading order Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

…ck like Ultralytics Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

…tics recognizer callable Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

UltralyticsAssistant · 2026-11-10T09:59:28Z

👋 Hello @onuralpszr, thank you for submitting a ultralytics/inference 🚀 PR! This automated message confirms your contribution was received, and an Ultralytics engineer will assist with the review. To ensure a seamless integration of your work, please review the following checklist:

✅ Define a Purpose: Clearly explain the purpose of your fix or feature in your PR description, and link to any relevant issues. Ensure your commit messages are clear, concise, and adhere to the project's conventions.
✅ Synchronize with Source: Confirm your PR is synchronized with the ultralytics/inference main branch. If it's behind, update it by clicking the 'Update branch' button or by running git pull and git merge main locally.
✅ Ensure CI Checks Pass: Verify all Ultralytics Continuous Integration (CI) checks are passing. If any checks fail, please address the issues.
✅ Update Documentation: Update the relevant documentation for any new or modified features.
✅ Add Tests: If applicable, include or update tests to cover your changes, and confirm that all tests are passing.
✅ Sign the CLA: Please ensure you have signed our Contributor License Agreement if this is your first Ultralytics PR; the CLA check will comment with signing instructions if needed.
✅ Minimize Changes: Limit your changes to the minimum necessary for your bug fix or feature addition. "It is not daily increase but daily decrease, hack away the unessential. The closer to the source, the less wastage there is." — Bruce Lee

For more guidance, please refer to our Contributing Guide. Don't hesitate to leave a comment if you have any questions. Thank you for contributing to Ultralytics! 🚀

UltralyticsAssistant

🔍 PR Review

_{Made with ❤️ by Ultralytics Actions}

Reviewed the OCR CLI, companion-session device selection, annotation, and segmentation-mask changes. Found one device-selection mismatch for native OpenVINO; the remaining reviewed paths showed no other concrete issue.

💬 Posted 1 inline comment

💡 MEDIUM src/model.rs:1485 With openvino-native and an Intel GPU, the detector's provider is "OpenVINO", so it misses the "OpenVINOExecutionProvider" arm and both companion OCR sessions fall through to CPU. This contradicts the documented claim that recognition runs on the detector's device and can significantly slow OCR on video or many detections. Handle the native OpenVINO backend for companion models, or make the CPU fallback explicit in the API/docs.

codecov · 2026-11-10T10:01:50Z

Codecov Report

❌ Patch coverage is 87.26542% with 95 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/ocr.rs	90.58%	61 Missing ⚠️
src/annotate.rs	57.33%	32 Missing ⚠️
src/cli/predict.rs	85.71%	1 Missing ⚠️
src/model.rs	80.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

…rs too Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

UltralyticsAssistant

🔍 PR Review 2

_{Made with ❤️ by Ultralytics Actions}

Since review 1, the native OpenVINO CPU fallback is now documented, resolving the earlier finding. The OCR changes otherwise look sound; the segmentation path still has a regression that drops detections whose masks are empty.

💬 Posted 1 inline comment

💡 MEDIUM src/postprocessing.rs:1034 If a post-NMS detection has no positive mask logits inside its nondegenerate box, apply returns None and this filter drops the detection; the later select_rows then removes its box, class, and confidence too. This makes segmentation results lose valid detections based solely on an empty mask. Keep the detection and emit its corresponding all-zero mask instead.

Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

…Rect like Ultralytics Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

UltralyticsAssistant

🔍 PR Review 3

_{Made with ❤️ by Ultralytics Actions}

Since review 2, the native OpenVINO CPU fallback is documented, addressing review 1; the empty-mask behavior was explained as intentional and is not repeated. I reviewed the current OCR API, companion-model loading, CLI integration, downloads, annotation, and documentation, and found no new actionable defects. LGTM.

Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

UltralyticsAssistant

🔍 PR Review 4

_{Made with ❤️ by Ultralytics Actions}

Since review 3, the earlier OpenVINO fallback remains documented and the empty-mask behavior remains settled. This pass found one new device-selection issue: TensorRT-only builds fall back to CPU for OCR companion sessions.

💬 Posted 1 inline comment

💡 MEDIUM src/model.rs:1470 With a build that enables tensorrt but not cuda—these are independent features in Cargo.toml—--device tensorrt:0 --ocr selects TensorRTExecutionProvider for the detector, but this arm is compiled out. The companion sessions then register no accelerator and run on CPU, contrary to the CUDA docs and potentially slowing video or images with many detections. Make the required CUDA provider available for TensorRT companion sessions, or explicitly require and document both features.

📋 Skipped 2 files (lock files, generated, images, etc.)

Cargo.lock
web/package-lock.json

…rRT builds Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

UltralyticsAssistant

🔍 PR Review 5

_{Made with ❤️ by Ultralytics Actions}

Since review 4, making tensorrt imply cuda addresses the companion-session device-selection finding. This pass found one remaining OCR rendering mismatch: confidence-only mode is ignored. The other reviewed OCR and CLI paths showed no additional actionable issues.

💬 Posted 1 inline comment

📝 LOW src/annotate.rs:825 When --show-labels=false --show-conf=true, OCR renders no label because this arm discards confidence too. The existing detection annotator shows confidence alone for the same option combination, and the PR says --show-conf applies to OCR. Render the confidence when labels are hidden, matching draw_detection.

📋 Skipped 2 files (lock files, generated, images, etc.)

Cargo.lock
web/package-lock.json

UltralyticsAssistant

🔍 PR Review 6

_{Made with ❤️ by Ultralytics Actions}

Since review 5, the TensorRT feature change addresses the prior device-selection issue, but the OCR confidence-only rendering finding remains open. This pass found no additional actionable issues.

💬 Posted 1 inline comment

📝 LOW src/annotate.rs:825 Since review 5 this confidence-only case remains unchanged: with show_labels=false and show_conf=true, OCR draws no label because this arm discards confidence too. The detection annotator renders confidence for the same option combination. Render the confidence here when labels are hidden so --show-conf works as documented.

📋 Skipped 2 files (lock files, generated, images, etc.)

Cargo.lock
web/package-lock.json

UltralyticsAssistant

🔍 PR Review 7

_{Made with ❤️ by Ultralytics Actions}

Since review 6, the TensorRT companion-device issue is addressed, and the confidence-only rendering behavior is settled as intentional. This pass found a new OCR crop-width rounding panic; otherwise, no additional actionable issues.

💬 Posted 1 inline comment

💡 MEDIUM src/ocr.rs:279 For crops wider than the minimum recognizer ratio, padded truncates while resized rounds up. For example, a 101×15 crop produces widths (324, 323), so tensor allocates only 323 columns and then slices 324, panicking during OCR on common non-integral aspect ratios. Round padded up as well so it is never smaller than resized.

📋 Skipped 2 files (lock files, generated, images, etc.)

Cargo.lock
web/package-lock.json

UltralyticsAssistant

🔍 PR Review 8

_{Made with ❤️ by Ultralytics Actions}

Since review 7, the crop-width concern is settled by the existing cap and the author’s explanation; the earlier TensorRT device-selection issue is addressed, and the OCR label behavior is settled as intentional. I found no remaining open findings or new actionable defects in the OCR, CLI, model-loading, annotation, download, or documentation changes. LGTM.

📋 Skipped 2 files (lock files, generated, images, etc.)

Cargo.lock
web/package-lock.json

UltralyticsAssistant

🔍 PR Review 8

_{Made with ❤️ by Ultralytics Actions}

Since review 7, the TensorRT feature change is addressed, the confidence-only rendering behavior is settled as intentional, and the crop-width concern was answered by the resize cap. I reviewed the OCR, CLI, model loading, downloads, and annotation changes; the new finding is avoidable per-session thread-pool overhead when OCR uses default thread settings.

💬 Posted 1 inline comment

💡 MEDIUM src/model.rs:1492 With the default num_threads=0, intra_threads resolves to every available core. TextRecognizer::new creates two companion sessions in addition to the detector, so CPU OCR (especially long-running video) can leave three independent per-session thread pools, with idle workers consuming CPU and resources. Use a shared pool or cap the companion sessions separately. (onnxruntime.ai)

📋 Skipped 2 files (lock files, generated, images, etc.)

Cargo.lock
web/package-lock.json

Superseded by new review

UltralyticsAssistant

🔍 PR Review 10

_{Made with ❤️ by Ultralytics Actions}

Since review 9, the per-session thread-pool concern is settled by the author’s measurements; no earlier finding remains open. I rechecked the OCR processing, companion-model loading, CLI integration, annotation, and downloads. No new actionable regressions found. LGTM.

📋 Skipped 2 files (lock files, generated, images, etc.)

Cargo.lock
web/package-lock.json

UltralyticsAssistant · 2026-11-10T15:16:32Z

🎉 PR #631 has been merged—thank you, @onuralpszr, for adding native scene-text OCR to the Rust API and predict --ocr, combining YOLO26-Text detection with PaddleOCR recognition.

“Alone we can do so little; together we can do so much.” — Helen Keller

A fitting thought for bringing text detection and recognition together in one OCR workflow.

onuralpszr added 11 commits November 10, 2026 11:31

Build instance masks like Ultralytics process_mask_native so they mat…

d66862f

…ch Python pixel for pixel Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

Drop segment detections whose mask is empty like Ultralytics

e23ebb4

Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

Add retina_masks and build segmentation masks at the inference size b…

48f3773

…y default like Ultralytics Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

Add Masks xy and xyn outlines traced like OpenCV findContours as in U…

a4f1bf3

…ltralytics Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

Scale mask outlines from the inference size to the image like Ultraly…

a9f9231

…tics scale_coords Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

Add scene text OCR with YOLO26-Text and PaddleOCR through TextRecogni…

63519f3

…zer and predict --ocr Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

Read OCR model outputs in place instead of copying them

9a93f3d

Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

Shrink the OCR module with shared batch building and simpler crops an…

6882637

…d reading order Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

Place OBB and OCR labels above the box and draw OCR label text in bla…

b3dd338

…ck like Ultralytics Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

Draw OCR labels at the Python size and keep each one at its box

ec62c67

Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

Let TextRecognizer read crops with a custom function like the Ultraly…

061fc6c

…tics recognizer callable Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

UltralyticsAssistant added documentation Improvements or additions to documentation enhancement New feature or request labels Nov 10, 2026

onuralpszr marked this pull request as draft November 10, 2026 09:59

UltralyticsAssistant reviewed Nov 10, 2026

View reviewed changes

onuralpszr added 3 commits November 10, 2026 13:05

Describe instance mask data as binary in the tasks example

4a94bac

Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

Describe Masks as binary masks with outlines on the original image

4eb940c

Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

Run OCR recognizers on the OpenVINO EP for native OpenVINO IR detecto…

a6748e1

…rs too Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

onuralpszr requested a review from UltralyticsAssistant November 10, 2026 10:18

UltralyticsAssistant reviewed Nov 10, 2026

View reviewed changes

Comment thread src/postprocessing.rs Resolved

Build input size masks through the same mask path as original size ones

1e421de

Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

onuralpszr changed the title ~~Add scene text OCR with YOLO26-Text and PaddleOCR~~ ultralytics-inference 0.0.58 Add scene text OCR with YOLO26-Text and PaddleOCR Nov 10, 2026

onuralpszr added 2 commits November 10, 2026 14:07

Merge main into feat-ocr

00e6bc2

Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

Read segmentation detector masks in OCR through a port of cv2.minArea…

6e4615b

…Rect like Ultralytics Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

onuralpszr commented Nov 10, 2026

View reviewed changes

Comment thread src/postprocessing.rs Resolved

Merge main into feat-ocr

18f6b0a

Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

onuralpszr requested a review from UltralyticsAssistant November 10, 2026 12:52

Bump ultralytics-inference to 0.0.58

04957be

Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

UltralyticsAssistant reviewed Nov 10, 2026

View reviewed changes

Update the docs for OCR, drawing options and 0.0.58

91f45ae

Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

onuralpszr requested a review from UltralyticsAssistant November 10, 2026 12:58

Update the issue template example versions

29ec014

Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

UltralyticsAssistant reviewed Nov 10, 2026

View reviewed changes

onuralpszr marked this pull request as ready for review November 10, 2026 14:15

Build cuda with the tensorrt feature so OCR reads on the GPU in Tenso…

cdb609d

…rRT builds Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

onuralpszr requested a review from UltralyticsAssistant November 10, 2026 14:20

UltralyticsAssistant reviewed Nov 10, 2026

View reviewed changes

onuralpszr requested a review from UltralyticsAssistant November 10, 2026 14:22

UltralyticsAssistant reviewed Nov 10, 2026

View reviewed changes

Comment thread src/annotate.rs Resolved

onuralpszr commented Nov 10, 2026

View reviewed changes

Comment thread src/annotate.rs Resolved

UltralyticsAssistant reviewed Nov 10, 2026

View reviewed changes

Comment thread src/ocr.rs Resolved

onuralpszr commented Nov 10, 2026

View reviewed changes

Comment thread src/ocr.rs Resolved

onuralpszr requested a review from UltralyticsAssistant November 10, 2026 15:07

UltralyticsAssistant previously approved these changes Nov 10, 2026

View reviewed changes

UltralyticsAssistant reviewed Nov 10, 2026

View reviewed changes

Comment thread src/model.rs Resolved

onuralpszr commented Nov 10, 2026

View reviewed changes

Comment thread src/model.rs Resolved

UltralyticsAssistant approved these changes Nov 10, 2026

View reviewed changes

onuralpszr merged commit 6816587 into main Nov 10, 2026
26 checks passed

onuralpszr deleted the feat-ocr branch November 10, 2026 15:16

Conversation

onuralpszr commented Nov 10, 2026 • edited by UltralyticsAssistant Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🛠️ PR Summary

🌟 Summary

📊 Key Changes

🎯 Purpose & Impact

Uh oh!

UltralyticsAssistant commented Nov 10, 2026

Uh oh!

UltralyticsAssistant left a comment

Choose a reason for hiding this comment

🔍 PR Review

Uh oh!

codecov Bot commented Nov 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

UltralyticsAssistant left a comment

Choose a reason for hiding this comment

🔍 PR Review 2

Uh oh!

Uh oh!

Uh oh!

UltralyticsAssistant left a comment

Choose a reason for hiding this comment

🔍 PR Review 3

Uh oh!

UltralyticsAssistant left a comment

Choose a reason for hiding this comment

🔍 PR Review 4

Uh oh!

UltralyticsAssistant left a comment

Choose a reason for hiding this comment

🔍 PR Review 5

Uh oh!

UltralyticsAssistant left a comment

Choose a reason for hiding this comment

🔍 PR Review 6

Uh oh!

Uh oh!

Uh oh!

UltralyticsAssistant left a comment

Choose a reason for hiding this comment

🔍 PR Review 7

Uh oh!

Uh oh!

Uh oh!

UltralyticsAssistant left a comment

Choose a reason for hiding this comment

🔍 PR Review 8

Uh oh!

UltralyticsAssistant left a comment

Choose a reason for hiding this comment

🔍 PR Review 8

Uh oh!

Uh oh!

Uh oh!

UltralyticsAssistant left a comment

Choose a reason for hiding this comment

🔍 PR Review 10

Uh oh!

Uh oh!

UltralyticsAssistant commented Nov 10, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

onuralpszr commented Nov 10, 2026 •

edited by UltralyticsAssistant

Loading

codecov Bot commented Nov 10, 2026 •

edited

Loading